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ABSTRACT 


The  shift  towards  qualifying  performance  and  accounting  for  results  has 
dramatically  changed  the  way  Government  executes  public  policy  objectives.  The  advent 
of  the  Government  Performance  and  Results  Act  (GPRA)  places  the  responsibility  for 
gathering  this  information  upon  each  Federal  activity  subject  to  its  provisions.  The  Army 
Small  Business  Innovation  Research  (SBIR)  program  must  now  find  a  way  to  qualify  its 
performance  and  determine  what  results  are  derived  from  a  program  that  expends  in 
excess  of  $100,000,000  annually  on  research.  This  thesis  analyses  Army  SBIR 
commercialization  rates  against  a  National  Science  Foundation  study  of  DoD  Fast  Track 
and  DoD  Control  Group  awards.  It  provides  an  objective  measure  of  program  results  that 
program  officials  can  use  to  submit  their  annual  GPRA  performance  reports.  The  thesis 
studied  37  SBIR  Phase  II  firms  and  established  a  performance  baseline.  The  thesis 
concludes  that  Army  SBIR  awards  are  outperforming  DoD  Fast  Track  and  DoD  Control 
Groups  in  the  critical  area  of  average  commercial  sales  per  award.  It  recommends  a 
reduced  focus  on  outside  investment  and  a  survey  strategy  that  uses  small  sample  sizes  to 
qualify  program  performance.  It  concludes  with  a  proposed  survey  instrument  that  Army 
SBIR  managers  can  use  to  capture  future  program  outcomes. 
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I.  INTRODUCTION 


A.  GENERAL 

“The  relative  weight  that  should  be  given  to  [agency]  goals  when 
evaluating  proposals  remains  unclear.”  (Ref.  1 :  p.7) 


The  Small  Business  Innovation  Research  (SBIR)  program  provides  significant 
opportunities  for  small  businesses  to  participate  in  Federal  Government  research  and 
development  efforts  every  year.  The  program  awards  contracts  to  small  companies  that 
demonstrate  in  a  competitive  proposal  process  that  they  can  perform  innovative  research 
and  development  that  serves  a  Department  of  Defense  (DoD)  need.  It  also  seeks  to  fund 
efforts  that  provide  important  commercial  applications  that  add  to  national  economic 
objectives.  The  method  DoD  uses  to  accomplish  this  goal  is  to  provide  funding  up  to 
$850,000  directly  to  each  award  winner  for  a  specific  research  topic. 

Since  its  inception  in  1982,  the'program  has  evolved  to  reflect  a  compilation  of 
agency  goals  indicative  of  a  public  program  with  multiple  stakeholders.  This  project  will 
elucidate  how  those  goals  impact  program  execution. 

1.  The  Problem 

All  Federal  agencies  with  extramural  budgets  for  research  and  development  are 

required  to  set  aside  2  1/2  %  of  their  total  research/research  and  development  funds  to 

support  the  SBIR  program.  The  presenting  problem  that  this  thesis  will  address  is  that 

the  allocation  of  precious  resource  funds  expended  on  this  program  are  not  adequately 

being  accounted  for  in  terms  of  results  in  accordance  with  the  Government  Performance 

Results  Act  of  1993.  Without  a  full  accounting  of  benefits  derived  from  the  program,  it  is 
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impossible  to  quantitatively  assess  whether  the  program  is  successful  in  meeting  the 
stated  goals  of  the  principle  stakeholders. 

2.  Potential  Solutions 

The  researcher  proposes  three  key  changes  to  the  Army  Research  Office’s 
standard  operating  procedures:  1)  Reduce  its  emphasis  on  outside  investment  as  an 
evaluation  criterion;  2)  Conduct  regular  program  surveys  and  ;  3)  Use  small  survey 
populations  to  develop  annual  program  results  reports. 

3.  What  Happens  if  the  Problem  Remains  Unanswered? 

If  the  program  continues  to  exist  in  its  current  form  without  the  controls  or 
feedback  loops  needed  to  assess  program  success,  the  difficulty  in  determining  a  return 
on  investment  for  the  allocation  of  Federal  research  and  development  funds  will  continue. 
This  potential  misallocation  of  resources  will  produce  an  opportunity  cost  to  the  taxpayer 
and  reduce  Government  efficiency. 

B.  OBJECTIVES 

The  objectives  of  this  project  are  to  determine  if  a  correlation  between  technology 
sales  and  commercialization  potential  exist  to  support  the  Army’s  current  evaluation 
criteria  and  to  measure  the  prospects  of  commercial  sales  on  the  technical  approach  taken 
by  prospective  firms.  It  also  seeks  to  validate  whether  current  procedures  are  effective  in 
satisfying  agency  goals  and  objectives  and  fulfilling  Congressional  intent.  The 
completed  thesis  will  build  upon  an  earlier  body  of  research  conducted  on  projects  from 
1994-1996  and  address  public  law,  acquisition  principles,  and  agency  specific  strategic 
plans. 

C.  RESEARCH  QUESTIONS 

The  following  research  questions  will  be  answered  by  the  thesis. 
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1.  Primary  Research  Question 

How  does  the  Army  define  and  measure  Small  Business  Innovation  Research 
(SBIR)  Program  success? 

2.  Subsidiary  Research  Questions 

•  What  are  the  Small  Business  Innovation  Research  (SBIR)  and  Fast  Track 
Programs? 

•  What  are  the  stated  goals  of  the  DoD  SBIR  Program? 

•  What  are  the  various  laws,  regulations,  and  policies  that  might  affect  the  SBIR 
and  Fast  Track  Programs? 

•  What  major  products  are  created  through  the  Army  SBIR  program  and  how 
does  the  Department  of  Defense  integrate  those  products  into  its  inventory? 

•  How  does  the  Department  of  the  Army  track  the  commercialization  of  SBIR 
products  through  the  program’s  life  cycle? 

•  Do  current  Department  of  the  Army  requirements  generation,  marketing, 
solicitation,  proposal  evaluation,  source  selection  and  surveillance  procedures 
lead  to  enhanced  program  success  and  greater  commercialization  of  products? 

•  What  changes  are  needed  to  the  Department  of  the  Army’s  SBIR  Program  to 
enhance  the  program’s  performance  as  it  relates  to  stated  goals? 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

1.  Scope 

The  scope  of  this  thesis  will  include:  (1)  an  in-depth  review  of  the  SBIR  program 
and  its  stated  goals,  (2)  an  evaluation  of  the  current  proposal  selection  process,  and  (3)  an 
objective  analysis  of  whether  the  current  system  is  optimizing  program  execution.  The 
thesis  will  conclude  with  proposed  recommendations  for  optimizing  program  execution. 

2.  Limitations 

The  research  will  be  limited  by  the  fact  that  the  Army  SBIR  Program  has  not 
undergone  any  exhaustive  analysis  of  its  effectiveness  in  producing  outcomes. 
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Furthermore,  due  to  the  limited  resources,  feedback  mechanisms  for  ascertaining 
conclusive  commercialization  results  via  product  or  process  tracking  for  all  Phase  II 
participants  are  not  available. 

3.  Assumptions 

The  following  assumption  will  be  used  in  preparing  the  thesis  report. 

a.  Any  reader  of  the  thesis  is  assumed  to  have  a  fundamental 
understanding  of  the  Defense  Department  and  Army  in  general,  and  at  least  a  surface 
familiarity  with  the  procedures  and  functions  associated  with  the  acquisition  community. 

b.  It  is  assumed  that  the  provisions  of  the  Small  Business  Innovation 
Research  Program  Reauthorization  Act  of  1999  will  remain  in  force  until  2008. 

c.  It  is  assumed  that  the  Army  SBER.  Program’s  standing  operating 
procedures  will  remain  fundamentally  unchanged  for  the  foreseeable  future. 

E.  DEFINITIONS  AND  ABBREVIATIONS 

A  complete  list  of  pertinent  acronyms  is  provided  in  the  Acronym  listing  in  the 
front  of  this  document.  Several  key  terms,  however,  are  shown  below. 

1.  Research  and  Development 

Any  activity  that  is:  (Ref.  2) 

(a)  A  systematic,  intensive  study  directed  toward  greater  knowledge  or 
understanding  of  the  subject  studied. 

(b)  A  systematic  study  directed  specifically  toward  applying  new  knowledge  to 
meet  a  recognized  need. 
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(c)  A  systematic  application  of  knowledge  toward  the  production  of  useful 
materials,  devices,  and  systems  or  methods,  including  design,  development,  and 
improvement  of  prototypes  and  new  processes  to  meet  specific  requirements. 

2.  Commercialization 

The  process  of  developing  markets  and  producing  and  delivering  products  for  sale 
(whether  by  the  originating  party  or  by  others),  as  used  here,  commercialization  includes 
both  Government  and  commercial  markets.  (Ref  2) 

3.  Fast  Track 

An  expedited  proposal  evaluation  process  that  requires  third  party  investment  for 
participation.  This  program  affords  research  firms  that  arrange  for  third  party  investment 
to  experience  significantly  higher  chances  of  contract  award  by  providing  opportunities 
to  leverage  matching  Federal  funds  of  between  $1  and  $4  for  every  $1  the  investor 
provides.  (Ref  2) 

4.  Extramural  Budget 

The  sum  of  the  total  obligations  of  research/research  and  development  minus 
amounts  obligated  for  Research/research  and  development  activities  by  employees  of  the 
agency  in  or  through  Government-owned,  Government-operated  facilities.  (Ref  2) 

5.  Funding  Agreement 

Any  contract,  grant,  or  cooperative  agreement  entered  into  between  any  Federal 
agency  and  any  small  business  concern  for  the  performance  of  experimental, 
developments,  or  research  work  funded  in  whole  or  in  part  by  the  Federal  Government. 
(Ref  2) 
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6.  Subcontract 


Any  agreement,  other  than  one  involving  an  employer-employee  relationship, 
entered  into  by  a  Federal  Government  funding  agreement  awardees  calling  for  supplies  or 
services  required  solely  for  the  performance  of  the  original  funding  agreement.  (Ref  2) 

7.  Small  Business  Concern 

A  small  business  concern  is  one  that,  at  the  time  of  award  of  Phase  I  and  Phase  II 
funding  agreements,  meets  the  following  criteria:  (Ref  2) 

(a)  Is  independently  owned  and  operated;  is  not  dominant  in  the 
field  of  operation  in  which  it  is  proposing;  has  its  principal  place  of  business  located  in 
the  United  States  and  is  organized  for  profit. 

(b)  Is  at  least  50  percent  owned,  or  in  the  case  of  a  publicly  owned 
business,  at  least  50  percent  of  its  voting  stock  is  owned  by  United  States  citizens  or 
lawfully  admitted  permanent  resident  aliens. 

(c)  Has,  including  its  affiliates,  no  more  that  500  employees. 
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II.  BACKGROUND 


A.  GENERAL 

In  order  to  facilitate  the  discussion,  we  must  first  explore  the  events  leading  to  the 
formulation  of  the  SBIR  Program.  This  chapter  will  specifically  address  the  introduction 
of  the  Small  Business  Innovation  Development  Act  of  1982;  the  key  provisions  imposed 
by  the  Act;  significant  developments  fostered  through  subsequent  Acts;  program  goals; 
the  organization  structure  and  mission  of  the  Army  Research  Office;  and  a  description  of 
the  Fast  Track  Program. 

B.  SBIR  PROGRAM  INTRODUCTION 

In  an  effort  to  support  the  inclusion  of  small  business  initiatives  in 
research/research  and  development,  Congress  established  the  Small  Business  Innovation 
Development  Act  in  1982  (Ref  3)  to  provide  increased  opportunities  for  small  businesses 
to  participate  in  research/research  and  development,  to  increase  employment,  and  to 
improve  U.S.  competitiveness. 

The  SBIR  program  was  scheduled  to  expire  on  October  1,  1988.  However, 
Congress  initially  extended  the  program  to  September  30,  1993.  The  reasons  given  for 
extending  SBIR  were  that  the  program  creates  new  jobs,  increases  productivity  and 
economic  growth,  helps  combat  inflation,  and  stimulates  exports.  (Ref  4,  p.3)  In 
extending  the  program,  Congress  acknowledged  that  small  business  had  not  been 
receiving  a  fair  share  of  Federal  research/research  and  development  dollars.  In  1992, 
Congress  enacted  the  Small  Business  Research  and  Development  Enhancement  Act 
which  again  extended  SBIR,  this  time  through  October  1,  2000.  President  Bush  signed 
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the  bill  into  law  on  October  28,  1992.  The  current  authorization  (Ref  5)  reauthorizes  the 
program  until  September  30, 2008. 

C.  KEY  PROVISIONS  OF  THE  ACT 

Federal  Agencies  with  research  and  development  budgets  exceeding  $100  million 
are  required  to  set  aside  a  portion  of  those  funds  to  support  the  program.  The  minimum 
funding  level  has  steadily  increased  with  each  successive  reauthorization.  The  original 
bill  required  1%  of  an  agency’s  budget.  Since  then,  the  proportion  has  grown  from 
1.25%  during  1987-1992  to  1.5%  in  1993  and  1994  to  2%  in  1995  and  1996.  Since  1997, 
at  least  2.5  percent  of  all  extramural  research  funds  exceeding  the  $100  million  threshold 
have  been  set-aside  for  SBIR.  (Ref  6,  p.46) 

D.  SIGNIFICANT  DEVELOPMENTS 

In  1992,  Congress  reauthorized  the  act  with  significant  modifications.  The  Small 
Business  Research  and  Development  Enhancement  Act  of  1992  applied  a 
commercialization”  standard  to  the  proposal  selection  process  to  encourage  the 
promulgation  of  research  with  commercial  market  potential.  The  act  defined 
commercialization  as  the  process  of  developing  markets  and  producing  and  delivering 
products  for  sale.  (Ref  7)  This  major  shift  in  focus  altered  how  proposals  would  be 
evaluated  in  the  future  by  placing  a  premium  on  a  factor  that,  at  times,  influences  the 
technical  approach  proposed  by  potential  offerors. 

A  second  major  change  within  the  framework  of  the  program  took  place  with  the 
1999  reauthorization.  The  Small  Business  Innovation  Research  Program  Reauthorization 
Act  of  1999,  added  reporting  requirements  that  supported  the  Government  Performance 
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and  Results  Act  of  1993.  It  specifically  mandated  that  agencies  include  as  part  of  their 
annual  performance  plan:  (Ref  8) 

a)  The  establishment  of  performance  goals  to  define  the  level  of  performance  to 
be  achieved. 

b)  The  expression  of  such  goals  in  an  objective,  quantifiable,  and  measurable 

form. 

c)  The  establishment  of  performance  indicators  to  be  used  in  measuring  outcomes 
from  each  program  activity. 

d)  A  benchmarking  system  for  comparing  actual  program  results  with  established 
performance  goals. 

e)  A  method  to  describe  the  means  to  be  used  to  verify  and  validate  measured 

values. 

These  reports  are  forwarded  to  the  Committee  on  Small  Business  of  the  Senate,  and  the 
Committee  on  Science  and  the  Committee  on  Small  Business  of  the  House  of 
Representatives. 

E.  TOPIC  GENERATION 

At  least  six  months  prior  to  the  release  of  the  SBIR  solicitation  in  May  of  each 
year,  Army  Research  Office-Washington  (ARO-W)  issues  a  call  for  topics  to  the  Labs 
and  Research  Development  and  Engineering  Centers  (RDEC)  to  be  considered  for 
inclusion  in  the  solicitation.  Each  LAB  or  RDEC  is  allocated  a  number  of  topics  based 
on  current  Army  guidance  and  expected  future  SBIR  budgets.  Topic  authors  must  be 
Army  civilian  or  Army  military  personnel  with  requisite  technical  expertise  and  be 
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assigned  to  the  cognizant  Lab  or  RDEC  or  an  associated  Program  Manager  (PM)  or 
Program  Executive  Office  (PEO).  The  topic  authors  in  the  Labs  and  RDECs  develop 
topics  that  address  their  respective  organizations’  objectives.  Topic  authors  should  also 
coordinate  with  their  relevant  battle  Lab  to  get  the  “user”  perspective.  Finally  the  topics 

submitted  must  be  in  accordance  with  the  Lab  and  RDEC  Technical  Director’s  priorities. 
(Ref  9,  p.10) 

F.  SBIR  PROGRAM  GOALS 

The  Small  Business  Innovation  Research  Program  Reauthorization  Act  of  1992 
forms  the  basis  of  the  current  program’s  execution.  In  the  accompanying  House  Report, 
Congress  noted  that  the  program  was  highly  successful  in  integrating  small  business  in 
Federal  research.  (Ref  4,  p.l)  It  further  noted  that  small  businesses  were  producing 
innovative  goods  and  services,  and  that  the  program  was  the  “catalyst  in  the  promotion  of 
research  and  development”  for  the  nation’s  high-technology  industries.  (Ref  4,  p.l)  The 
report  stressed  the  ability  of  small  firms  to  capitalize  on  scientific  and  technical 
innovations  and  quickly  transform  those  findings  into  new  products  and  services  thereby 
enhancing  national  economic  growth.  (Ref  4,  p.4)  The  legislation  addressed  the 
congressional  concern  that  although  small  businesses  were  the  most  productive  source  of 
significant  innovation  in  the  nation,  their  share  of  Federally  funded  research  and 
development  efforts  was  understated  in  terms  of  contribution.  (Ref  7) 

Congress  designated  four  major  goals  (Ref  6,  p.46)  in  authorizing  the  program  to: 

1)  stimulate  technological  innovation; 

2)  use  small  business  to  meet  Federal  research  and  development 

needs; 
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3)  foster  and  encourage  participation  by  minority  and  disadvantaged 
persons  in  technological  innovation;  and 

4)  increase  private-sector  commercialization  of  innovations  derived 
from  Federal  research  and  development. 

G.  SBIR  PROGRAM  PHASES 

The  SBIR  program  is  subdivided  into  three  distinct  phases.  Although  each 
agency  participating  in  the  program  has  the  autonomy  to  administer  the  program 
differently,  the  basic  parameters  which  define  each  phase  are  essentially  the  same.  For 
illustration,  the  researcher  will  outline  the  phases  as  they  relate  to  Army  program 
execution. 

1.  Phase  I 

Phase  I  is  a  feasibility  study  of  approximately  six  months  in  duration.  Only 
proposals  which  respond  to  specific  Army  topics  published  in  the  second  of  two  annual 
DoD  SBIR  Program  Solicitations  are  eligible  for  Phase  I  consideration.  Funding  for 
Phase  I  is  not  to  exceed  $70,000,  with  an  additional  $50,000  available  as  a  separately 
priced  option  if  that  provision  it  is  included  with  the  initial  proposal.  The  option  may  be 
exercised,  at  the  Army’s  discretion,  to  fund  start-up  Phase  II  activities  for  those  Phase  I 
projects  which  have  been  selected  for  Phase  II  negotiation  and  award.  (Ref  9,  p.4) 

2.  Phase  II 

There  is  no  solicitation  for  Phase  II  proposals,  but  successful  Phase  I  firms  may 
be  invited  by  the  Army  to  participate  in  Phase  II  of  the  SBIR  Program  where  the  primary 
R/R&D  work  is  conducted.  Phase  II  awards  are  two  year  efforts  that  are  funded 
incrementally  up  to  a  maximum  of  $750,000.  (Ref  9,  p.4) 
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3.  Phase  III 


Successful  Phase  m  projects  result  in  technologies,  products,  or  services  that  can 
be  marketed  to  Government  or  commercial  customers  outside  the  SBIR  Program.  This 
commercialization  phase  is  the  ultimate  goal  of  each  SBIR  effort.  By  law,  Phase  m 
efforts  can  not  incorporate  SBIR  funds.  (Ref  9,  p.5) 

H.  ARMY  RESEARCH  OFFICE  ORGANIZATIONAL  STRUCTURE 

1.  Organizational  Structure 

The  Army  Research  Office-Washington  (ARO-W),  a  subordinate  office  of  the 
Army  Research  Office,  is  the  parent  organization  of  the  Army’s  SBIR  effort.  It  is 
structured  with  two  distinct  functions:  ARO-W  (A)  manages  the  SBIR  program  and 
ARO-W  (B)  monitors  and  participates  in  DoD-wide  science  and  technology  activities 
that  leverage  the  Army's  investment  in  basic  research. 

2.  Mission 

The  mission  of  ARO-W  (A)  is  to  serve  as  the  liaison  between  ARO  Headquarters 
and  Army  Materiel  Command  (AMC).  The  office’s  primary  function  is  to  execute 
Army-wide  technology  development  in  partnership  with  industry  and  academia  through 
three  unique  programs:  The  Army  Small  Business  Innovation  Research  (SBIR)  Program, 
the  Army  Small  Business  Technology  Transfer  (STTR)  Program,  and  the  Advanced 
Concepts  and  Technology  Program  (ACT  II).  Each  program  has  unique  goals  and 
operating  parameters.  The  ARO-W  acts  as  the  executive  agent  for  all  Army  SBIR 
activities  and  coordinates  program  actions  within  the  Army's  laboratories  and  RDECs. 
(Ref  10) 
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3.  Proposal  Evaluation 

Proposal  submissions  are  evaluated  on  scientific,  technical,  and  commercial  merit. 
Because  of  the  proprietary  nature  of  the  criteria  and  associated  weighting,  that 
information  will  not  be  disclosed  as  a  component  of  this  report.  The  researcher 
acknowledges  that  commercial  potential  may  have  a  significant  impact  upon  approval  of 
Phase  II  awards. 

4.  SBIR  Marketing  Assistance 

The  Army  SBIR  office  expends  considerable  effort  to  publicize  program 
innovations.  They  assist  firms  in  marketing  their  products  through  the  annual  SBIR 
Phase  III  success  booklet  publication.  This  booklet  is  distributed  throughout  the  Army;  at 
National,  Regional,  and  State-level  SBIR  Conferences  and  workshops.  In  many 
instances,  representatives  of  large  businesses  and  venture  capital  firms  attend  such  events 
to  “network”  with  Army  SBIR  firms.  SBIR  Phase  III  success  stories  are  posted  on  the 
ARO-W  website  and  are  routinely  provided  to  SBA  and  in  response  to  Congressional- 
related  request. 

I.  FAST  TRACK  PROGRAM 

The  DoD  initiated  the  Fast  Track  program  in  an  effort  to  improve  the  rate  of  SBIR 

commercialization.  Starting  with  the  1996  solicitations,  DoD  initialed  a  two  year  pilot 

policy  -  the  SBIR  Fast  Track  -  under  which  SBIR  projects  that  attract  matching  funds 

from  third-party  investors  have  a  significantly  higher  probability  of  SBIR  award. 

Additionally,  the  SBIR  program  office  expedites  proposal  evaluation  and  processing  to 

reduce  the  delay  in  reaching  the  market.  The  purpose  of  the  Fast  Track  policy  is  to 

concentrate  SBIR  funds  on  those  research  and  development  projects  most  likely  to  result 

in  viable  new  products  that  DoD  and  other  will  buy.  (Ref  6,  p.52) 
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At  the  conclusion  of  the  initial  pilot  period,  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology  extended  the  Fast  Track  pilot  for  two  additional  years, 
because  of  the  promising  early  results.  The  Under  Secretary  also  directed  an  independent 
analysis  of  Fast  Track.  The  National  Research  Council  (NRC)  was  asked  to  conduct  that 
analysis.  (Ref  6,  p.51)  The  study’s  focus  covered  a  total  of  379  DoD  projects  selected 
from  among  the  2,574  that  received  Phase  II  awards  during  1992 -1996.  The  sample 
contained  award  data  from  every  agency  operating  under  the  DoD  organizational 
structure. 

The  NRC  analysis  of  results  concluded  that  the  Fast  Track  program  was  clearly 
outperforming  DoD  Control  Group  awards.  The  areas  of  sales,  additional  developmental 
funding  and  expected  sales  were  all  sighted  as  having  higher  aggregate  values.  (Ref  6, 
P-44) 

J.  PRIOR  STUDIES 

In  1991,  the  General  Accounting  office  (GAO)  conducted  a  study  across  all 
Federal  agencies  participating  in  the  program  to  evaluate  the  aggregate  commercial 
trends  of  products  that  were  studied  and  funded  through  Phase  II.  The  1991  survey 
questionnaire  was  sent  to  all  Phase  II  awardees  from  the  first  four  years  of  Phase  II 
program  awards:  1984-1987.  The  GAO  selected  the  earliest  recipients  because  studies  by 
technology  experts  concluded  that  the  incubation  period  for  a  concept  to  progress  to  a 
commercial  product  was  five  to  nine  years.  (Ref  6,  p.51) 

K.  GOVERNMENT  PERFORMANCE  AND  RESULTS  ACT 

Congress  passed  the  Government  Performance  and  Results  Act  (GPRA)  as  part  of 
a  legislative  framework  to  instill  performance-based  management  in  the  Federal 
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Government.  The  Results  Act  established  a  management  system  to  set  agency  goals  for 
program  performance  and  to  measure  results  against  those  goals.  In  enacting  the  Act, 
Congress  and  the  administration  realized  that  the  transition  to  result-oriented 
management  would  not  be  easy.  For  that  reason,  the  Act  provided  for  a  phased  approach 
to  implementation.  (Ref  11) 

Implementing  the  GPRA  in  a  research  environment  is  particularly  challenging.  In 
the  past,  research  agencies  have  cited  numerous  barriers  to  their  efforts  to  establish 
results-oriented  goals  and  measures.  (Ref  12)  The  barriers  include  problems  in  obtaining 
data  to  demonstrate  results,  accounting  for  factors  outside  of  the  agency’s  control  that 
affect  results,  and  dealing  with  the  long  time  periods  often  needed  to  see  results. 

Over  the  past  several  years,  the  Government  Accounting  Office  has  issued  reports 
that  identified  practices  for  improving  GPRA  implementation  in  Federal  agencies.  These 
reports  have  focused  on,  among  other  things,  overcoming  agency  specific  barriers, 
improving  the  usefulness  of  annual  performance  plans,  and  measuring  program  results 
that  are  under  limited  Federal  control.  These  reports  point  out  the  depth  and  scope  of 
management  practices  needed  to  successfully  implement  performance-based  management 
as  envisioned  under  the  GPRA.  (Ref  11) 

L.  SUMMARY 

This  chapter  provided  essential  background  information  on  the  SBIR  Program 
designed  to  enhance  the  readers’  understanding.  It  focused  primarily  on  the  origins  of  the 
program,  significant  developments,  and  basic  structural  framework.  The  chapter 
concluded  with  a  short  synopsis  of  the  Government  Performance  and  Results  Act  and  its 
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influence  on  reporting  results.  The  following  chapter  will  discuss  the  research 
methodology  used  to  gather  data  for  analysis. 
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III.  RESEARCH  METHODOLOGY 


A.  GENERAL 

The  research  methodology  used  to  gather  data  for  the  thesis  centered  on  two 
primary  means:  personal  interviews  with  the  program  manager  and  program  executors 
and  a  survey  questionnaire  focused  on  previous  award  recipients.  Chronologically,  there 
was  an  initial  interview  that  launched  the  research  process,  followed  by  the  transmission 
of  the  survey  questionnaire,  and  then  the  conduct  of  an  additional  personal  interview  with 
the  Army  Research  Office  SBIR  Program  Manager.  Each  of  these  steps  is  discussed 
below. 

B.  INITIAL  PERSONAL  INTERVIEW 

The  purpose  of  the  initial  interview  was  to  ascertain  whether  an  objective  analysis 
of  the  SBIR  Program  was  feasible  and  to  determine  how  the  Army  Research  Office  was 
obtaining,  assessing,  and  reporting  the  performance  of  contractors  that  had  participated  in 
the  program.  The  goal  of  this  phase  was  to  set  the  direction  for  further  research 
exploration. 

During  the  initial  interview,  (Ref  13)  a  program  support  contractor  supporting  the 
Army  Research  Office  SBIR  effort,  provided  his  insight.  He  was  instrumental  in 
assessing  the  state  of  the  program  and  offering  a  unique  perspective  into  the  program  that 
shaped  the  direction  of  the  project. 

C.  SURVEY  QUESTIONNAIRE 

The  survey  was  conducted  draw  a  parallel  between  prior  studies  and  current 
practice.  The  goal  was  to  produce  a  baseline  for  comparison  that  could  be  used  to 


measure  program  results  attributable  to  previous  changes  in  organizational  practices.  The 
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method  for  selecting  the  survey  recipients  (target  audience),  and  constructing  the  actual 
questionnaire  (survey  design)  are  presented  in  the  following  paragraphs. 

1.  Target  Audience 

Given  the  popularity  of  the  SBIR  Program,  the  researcher  decided  that  the  best 
strategy  for  conducting  the  survey  would  be  to  analyze  Phase  II  awards  for  both  Fast 
Track  and  non-Fast  Track  participants  in  a  given  period.  A  target  audience  was  selected 
using  only  those  firms  that  received  a  Phase  II  award  from  1996  through  1998.  The 
researcher  determined  that  only  those  firms  that  had  matriculated  through  Phase  II  would 
be  able  to  provide  any  return  on  investment  for  the  taxpayer  via  commercialization  of 
technology  derived  from  the  program.  The  critical  factor  in  pursuing  such  a  focus  was  to 
preempt  any  claim  of  sampling  error  or  survey  variance  by  surveying  a  statistically 
significant  number  of  firms. 

The  target  audience  analysis  rationale  was  directly  attributable  to  program 
guidelines  that  allow  the  ARO  to  invite  only  selected  firms  to  negotiate  Phase  I  interim 
and  Phase  II  awards.  (Ref  14)  Phase  I  interim  awards  were  excluded  because  of 
database  constraints  that  limited  their  validity.  The  1996  date  reflected  the  last  fiscal  year 
in  which  SBIR  program  research  proposals  were  evaluated  in  the  National  Science 
Foundation  study.  (Ref  6)  Finally,  the  latter  date  represented  the  last  year  a  firm  would 
have  had  to  begin  research  in  order  to  complete  the  Phase  II  effort  before  moving  into 
Phase  III.  This  cutoff  date  was  further  supported  by  analysis  of  the  so-called  two-year 
“incubation”  period  after  Phase  II  completion  that  firms  experience  before  any 
innovations  can  be  brought  to  the  marketplace.  (Ref  15) 
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A  search  of  the  SBIR  Award  Database  at  the  Office  of  Small  and  Disadvantaged 
Business  Utilization  for  the  Office  of  the  Secretary  of  Defense  served  as  the  starting  point 
for  identifying  specific  survey  participants.  (Ref  6)  The  researcher  identified  324  Phase 
II  awardees  that  fit  the  target  audience  profile.  A  sample  population  of  65  firms  was 
randomly  selected  from  the  SBA  Technet  database  for  the  study.  An  attempt  was  then 
made  to  telephonically  contact  each  firm  identified  on  the  list  in  order  to  determine  two 
points:  1)  if  the  firm  was  still  in  existence  and  2)  to  ascertain  who  in  the  firm  would  serve 
as  the  point  of  contact  for  the  questionnaire.  The  researcher  hoped  that  by  phoning  each 
firm,  locating  the  original  researcher  listed  in  the  database,  and  explaining  the  concept  of 
the  project  before  sending  the  questionnaire  would  serve  to  raise  the  response  rate.  Of 
the  65  firms  on  the  initial  target  audience  list,  49  firms  (75%)  were  contacted  and 
identified  as  survey  recipients.  Sixteen  firms  (25%)  identified  on  the  target  audience  lists 
were  deleted  due  to  bankruptcy,  restructuring,  loss  of  internal  documentation,  merger,  or 
an  unwillingness  to  participate.  Of  the  49  surveys  transmitted,  37  surveys  were  returned 
yielding  a  response  rate  of  76%. 

2.  Survey  Sample  Size 

Given  the  responsiveness  noted  above,  it  is  appropriate  to  ask  whether  the  sample 

was  large  enough  so  as  to  mitigate  or  eliminate  any  potential  clams  of  sampling  error.  In 

other  words,  was  the  survey  sample  of  sufficient  size  to  provide  a  clear  and  reliable 

indication  of  the  responses  that  would  have  been  given  if  the  entire  Phase  II  award 

population  had  been  surveyed?  Johnson  (Ref  17)  states  that  “when  [a]  sampled 

distribution  lacks  symmetry,  n  (the  number  of  observations)  may  have  to  be  quite  large 

(maybe  50  or  more)  before  the  normal  distribution  provides  a  satisfactory  approximation” 

for  the  probability  distribution  of  the  mean.  For  the  analysis  contained  in  the  next 
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chapter,  the  author  assumes  that  the  responses  presented  for  analysis  in  Chapter  IV  are 
normally  distributed. 

The  researcher  used  a  combination  of  factors  to  determine  the  minimum  required 
sample  size  using  the  largest  standard  deviation  for  the  three  scaled  sub-components 
contained  in  the  survey.  When  combined  with  a  maximum  desired  error  factor  of 
+/-.30  points  from  the  sample  mean,  a  confidence  coefficient  of  80%  (1.65),  and  standard 
deviation  for  the  recorded  responses  of  1.0836,  the  computation  yielded  a  sample  size  of 
36  completed  surveys.  A  total  of  37  surveys  were  collected  and  analyzed. 

3.  Survey  Design 

The  researcher  had  two  concerns  when  constructing  the  survey  questionnaire. 
The  first  was  to  design  a  survey  that  produced  objective  results  that  would  allow  for 
statistical  analysis  of  the  SBIR  Program  and  commercialization  rates.  The  second  was  to 
pose  the  questions/statements  in  such  a  way  as  to  eliminate  potential  bias  in  the  results. 
A  survey  questionnaire,  as  distributed  to  the  target  audience,  is  contained  in  Appendix  A. 

Given  the  concerns  of  the  researcher,  the  questionnaire  was  designed  with  three 
sections.  The  first  section  addressed  specific  historical  data  for  each  contract  the  firm 
undertook.  The  purpose  of  this  was  to  confirm  that  the  information  listed  on  the  DoD 
Under  Secretary  of  Defense  Website  database  matched  the  SBA  Technet  database.  The 
concept  was  to  provide  preliminary  check  of  the  validity  of  the  information  contained  in 
the  database  to  ensure  that  the  comments  and  responses  provided  by  the  firm  were 
credible. 

The  second  section  aimed  to  gather  additional  information  that  would  facilitate 
further  statistical  analysis  through  the  correlation  of  data  derived  from  DoD  SBIR 
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records.  The  final  section’s  focus  was  to  gain  an  understanding  of  each  firm’s 
perspective.  It  specifically  sought  to  gain  insight  into  how  each  firm  was  approaching  the 
program  and  how  various  approaches  were  influenced  by  the  firm’s  beliefs. 

D.  SECONDARY  INTERVIEWS 

Secondary  Interviews  were  conducted  two  months  after  survey  initiation.  The 
purpose  of  the  secondary  interviews  was  to  assess  what  policies  key  decision  makers 
deemed  essential  to  qualifying  the  results  of  the  program.  The  underlying  intent  was  to 
ascertain  whether  they  viewed  a  reporting  mechanism  as  necessary  given  the 
overwhelming  Congressional  support  for  the  program. 

E.  SUMMARY 

This  chapter  provided  an  in-depth  discussion  of  the  research  methodology  used  to 
develop  the  survey  instrument  used  to  gather  information  for  the  thesis.  It  identified  the 
purpose  of  the  initial  interview  and  the  rationale  supporting  survey  instrument 
development.  Additionally,  it  identified  the  need  for  an  assessment  of  the  program  from 
a  policy-making  perspective  to  ascertain  how  program  results  were  viewed  from  a  top- 
down  perspective.  The  next  chapter  will  present  a  statistical  examination  of  the  collected 
data  and  the  summarized  results  of  both  sets  of  interviews. 
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IV.  DATA  PRESENTATION  AND  ANALYSIS 


A.  INTRODUCTION 

This  chapter  presents  the  results  of  a  survey  administered  to  SBIR  Phase  II 
participants.  The  target  audience’s  responses  are  displayed  in  graphical  form  for  ease  of 
interpretation.  The  data  and  subsequent  analysis  presented  in  this  section  will  illuminate 
how  funding  and  program  solicitation  dynamics  impact  program  results.  This  chapter 
will  also  discuss  the  economic  impact  of  program  outcomes  for  technologies  and 
products  developed  as  a  direct  result  of  the  SBIR  program. 

The  SBIR  research  survey  instrument  was  based  on  a  previous  survey  that  the 
GAO  conducted  in  1992  and  a  National  Science  Foundation  Paper.  (Ref  6)  New 
questions  for  the  survey  instrument  were  developed  as  a  result  of  dialog  between  the 
researcher  and  the  Army  Research  Office  as  well  as  from  conversations  with  current 
SBIR  Phase  II  participants.  The  final  survey  was  reviewed  and  subsequent  changes  made 
with  the  assistance  of  a  survey  instrument  expert  at  the  Naval  Postgraduate  School.  (Ref 
18)  The  final  survey  reflected  high  face  validity  between  intended  and  perceived 
meaning  of  the  questions. 

The  SBIR  survey  was  sent  to  49  Phase  II  program  participants  during  a  three- 
month  period  starting  February  2001.  The  survey  yielded  a  76%  percent  response  rate 
with  37  firms  meeting  the  feedback  closing  date.  Participation  was  voluntary  with  the 
condition  that  individual  survey  responses  would  be  strictly  confidential.  Table  4.1 
represents  the  distribution  of  survey  respondents.  As  far  as  can  be  determined,  the  survey 
sample  appears  to  be  representative  of  the  entire  award  population. 
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Fiscal  Year  of 
Award 

Target  Audience 

Survey  Respondents 

Percentage  of 
Respondents 

FY  96 

16 

12 

75% 

FY  97 

19 

16 

84% 

FY  98 

14 

9 

64% 

Total 

49 

37 

76% 

Table  4. 1 .  Distribution  of  Survey  Respondents. 
(Source:  Designed  by  researcher) 


B.  GROUP  COMPARISON 

The  Army  SBIR  contract  awards  represent  a  cross  section  of  awards  given  over  a 
three  year  period  from  1996  to  1998.  The  firms  selected  for  participation  were  randomly 
chosen  from  the  324  contract  awards  negotiated  during  that  period. 

The  survey  data  collected  was  analyzed  against  two  groups  for  a  basis  of 
comparison.  The  first  group  of  awards,  1996  DoD  Fast  Track  award  winners,  consisted 
of  firms  that  qualified  under  the  Fast  Track  program  for  enhanced  commercial  potential. 
This  group  received  special  evaluation  consideration  and  expedited  proposal  processing. 
These  awards  represented  the  best  in  class  because  of  an  optimal  combination  of  sound 
technological  approach,  researcher  credentials,  and  commercial  potential  as  validated  by 
third  party  investment  funding. 

The  DoD  control  group  represented  a  sample  of  all  SBIR  contracts  across  the 
DoD  spectrum  for  fiscal  1996.  They  embody  all  of  the  salient  characteristics  of  the 
research  projects  the  program  seeks  to  fund  and  constitute  a  baseline  of  what  the  average 
award  should  be. 

C.  RESPONSES  TO  DEVELOPMENTAL  FUNDING  AND  SALES  QUESTIONS 

In  this  survey,  firms  were  asked  to  provide  quantifiable  business  outcomes  as  a 
direct  result  of  their  specific  project.  Developmental  funding  and  sales  questions  asked 
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firms  to  describe  the  extent  to  which  they  were  offering  products  developed  under  the 
program  to  the  commercial  market. 

1.  What  Products  Are  Going  to  Market? 

The  SBIR  program  affords  firms  the  opportunity  to  produce  a  myriad  of  products 
and  services  through  innovative  research.  Figure  4.1  displays  the  types  of  products  each 
surveyed  group  produces. 


□New  or  Improved 
Service  Capability 

□  Process  Technology 

□Final  Hardware  :  | 

Product  | 

□Intermediate  Hardware 
{  Product  or  Component  i 

■  Software 

i  BNo  Commercial 
Applications  Planned  • 

Figure  4. 1  Distribution  of  Intended  Commercial  Product. 

(Source:  Designed  by  the  researcher) 

When  compared  against  Fast  Track  awardees  and  the  DoD  Control  Group,  the 
Army  SBIR  group  is  heavy  weighted  towards  software  applications  and  final  hardware 
products.  This  phenomenon  is  indicative  of  two  key  themes:  a  solicitation  process  that 
promotes  material  solutions  to  army  requirements  and  a  continuous  push  to  digitize  the 
force. 


100% 


DoD  Fast  Army  DoD 

Track  SBIR  Control 

Awards  Group 
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2.  How  do  SBIR  Firms  Approach  Marketing? 

A  marketing  strategy  can  positively  or  negatively  affect  a  firm’s  ability  to  move  a 
product  to  market.  Some  of  the  best  innovations  “die  on  the  vine”  because  they  have  sat 
idle  on  the  shelf  without  a  mechanism  to  move  from  the  developer  to  the  customer. 
Completing  research  and  marketing  a  product  can  be  a  challenge  for  some  firms  because 
of  the  disparate  skill  sets  needed  to  complete  marketing  tasks.  Figure  4.2  illustrates  how 
firms  approached  marketing  from  a  company  perspective  with  regard  to  staff  hiring 
practices. 


M  Not  Needed 
H  Completed 
H  Underway 
■  Planned 

DoDFast  Army  SBIR  DoD  Control 
Track  Awards  Group 

Figure  4.2  Marketing  Staff  Hiring  Strategy 
(Source:  Designed  by  the  researcher) 

Of  the  37  Army  SBIR  firms  surveyed,  17  decided  that  there  was  a  need  for  a 
marketing  staff  to  facilitate  interest  in  their  product.  Although  this  closely  mirrors  78% 
of  Fast  Track  firms,  the  difference  between  those  two  groups  can  be  found  in  the 
percentage  of  firms  having  actually  started  the  hiring  process.  Fast  Track  firms  are  much 
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more  deliberate  in  their  efforts  to  bring  in  expertise  when  the  need  is  identified.  The 
percentage  of  firms  that  have  started  or  completed  the  marketing  staff  hiring  process  are 
57%  and  24%  for  Fast  Track  awardees  and  Army  SBIR  firms  respectively.  This  is  most 
likely  attributed  to  the  influence  of  the  outside  investment  needed  to  qualify  for  Fast 
Track  consideration.  Conversely,  Army  SBIR  firms  move  at  a  much  slower  pace.  This 
may  serve  to  hinder  a  firm’s  ability  to  realize  commercial  market  sales. 

3.  SBIR  Product  Consumption 

An  analysis  of  Army  SBIR  sales  data  conclusively  demonstrates  that  Army- 
proposals  are  developed  with  a  narrow  focus  on  military  applications.  As  a  group,  DoD 
purchased  85%  of  the  total  sales  generated  from  Army  SBIR  funded  technology.  Figure 
4.3  depicts  sales  by  sector  of  the  three  groups  and  clearly  reveals  that  DoD  is  the  prime 
beneficiary  of  Army  SBIR  program  objectives. 

100% 

80% 

60% 

40% 

20% 

0% 

DoD  Fast  Army  DoD 

Track  SBIR  Control 

Awards  Group 

Figure  4.3  Customer  Sales  by  Market 

(Source:  Designed  by  the  researcher) 
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4.  Average  Total  Sales 

Average  total  sales  data  per  award  is  a  key  performance  metric.  This  figure 
represents  an  empirical  measure  that  can  assist  program  managers  in  objectively 
assessing  program  success  via  a  return  on  investment  analysis.  Figure  4.4  graphically 
depicts  how  Army  SBIR  awards  have  performed  against  Fast  Track  awardees  and  the 
DoD  Control  Group. 


S  SI 40.000 
;  5120,000 
SI  00,000 
SS0.000 
S60.000 
S40.000 

$20,000 

so . 


Figure  4.4  Average' Total  Sales 
(Source:  Designed  by  the  researcher) 

The  exceptional  performance  of  Army  SBIR  firms  is  an  interesting  outcome  of 
the  research.  As  a  group.  Army  SBIR  firms  outperformed  DoD  Fast  Track  firms  by  21% 
and  the  DoD  Control  Group  by  164%  per  average  total  sales  per  award.  These  findings 
are  particularly  impressive  given  that  the  DoD  consumes  85%  of  all  Army  SBIR 
products. 


DoD  Fast  Track  Army  SBIR  Awards  DoD  Control  Group 
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Ten  firms  surveyed  commented  that  Army  projects  had  clear  linkages  to  Army 
and  DoD  requirements  that  created  a  ready  market  for  their  products.  The  availability  of 
this  market  influenced  nine  of  those  ten  firms  to  principally  focus  on  rapidly  developing 
their  technologies  to  conform  to  DoD  requirements  first  and  then  seek  to  exploit 
commercial  opportunities.  When  this  commitment  is  coupled  with  the  aforementioned 
DoD  market  dominance,  they  create  a  synergy  that  magnifies  program  return  on 
investment. 

D.  RESPONSES  TO  OTHER  COMPANY  AND  PROJECT  RELATED 

INFORMATION 

Fostering  ground-breaking  research  is  a  key  tenet  of  the  SBIR  program.  Given 
the  responses.  Army  SBIR  firms  overwhelmingly  conveyed  that  they  would  have  not 
dedicated  resources  and  effort  to  study  Army  topics.  As  a  group,  16  Army  SBIR  firms 
responded  that  they  would  “Definitely  Not”  have  explored  the  topic  they  received 
funding  to  study.  Another  13  stated  that  they  would  “Probably  Not”  have  opted  to 
explore  their  specific  topic.  Those  two  subgroups  combined  represented  80%  of  the 
responses. 

When  compared  against  Fast  Track  awardees,  the  differences  between  the  groups 
are  significant.  Only  13%  of  all  Fast  Track  awardees  stated  that  they  would  “Definitely 
Not”  or  “Probably  Not”  have  studied  the  topic  for  which  they  received  funding.  This  fact 
is  a  clear  indication  that  research  firms  see  little  commercial  value  in  the  technologies  the 
Army  wants  researched.  When  asked  why  they  would  not  have  researched  their  specific 
topic,  1 1  firms  commented  that  they  believed  the  purpose  of  the  research  was  to  meet  a 
specific  Army  research  requirement  that  did  not  have  any  “true”  civilian  applications. 
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^  Definitely  Not 
^Probably  Not 
®  Uncertain 
^Probably  Yes 
®  Definitely  Yes 


DoD  Fast  Army  SBIR  DoD  Control 
Track  Awards  Group 


Figure  4.5  Probability  of  a  Firm  Independently  Researching  a  SBIR  Topic 
(Source:  Designed  by  researcher) 


E.  PROGRAM  PERCEPTIONS  FROM  SBIR  PHASE  II  FIRMS 

In  the  final  portion  of  the  survey,  firms  were  asked  to  provide  their  opinions  of 
various  aspects  affecting  their  overall  perception  of  the  program.  A  five-point  Likert 
scale  was  used  to  formulate  an  assessment  of  how  each  firm  felt  about  third  party 
investment,  technical  approach,  and  marketing.-  Under  this  approach,  one  was  selected  to 
represent  the  highest  score  and  five  the  lowest.  The  questions  asked  each  firm  to 
determine  the  extent  to  which  they  agreed  or  disagreed  with  specific  principles  affecting 
their  proposal’s  ability  to  satisfy  Army  SBIR  solicitation  requirements.  The  scale  used 
was  as  follows: 

1  -  Strongly  Agree 

2  -  Agree 

3  -  Neither  Agree  nor  Disagree 

4  -  Disagree 

5  -  Strongly  Disagree 


30 


Scores  closer  to  one  (1)  mean  that  responses  displayed  a  high  degree  of 
association  to  the  proposed  statement.  Scores  closer  to  five  (5)  mean  that  responses 
displayed  a  high  degree  of  association  to  the  alternative  hypothesis. 

The  summarized  raw  responses  to  each  of  the  three  statements  in  the  Other 
Company  and  Project  Related  section  questionnaire  are  provided  on  the  following 
subsections.  Each  paragraph  lists  the  statement  posed  to  Phase  II  firms  and  presents  a 
graphical  depiction  of  the  summarized  responses.  The  mean  of  the  responses,  the 
standard  deviation,  and  the  Confidence  Interval  (Cl)  range  are  all  described  in  detail 
below  each  graph. 

The  scaled  responses  are  interpreted  for  favoring  or  disfavoring  the  impact  of 
investment  funding  on  program  participation  and  proposal  submission.  A  narrative 
analysis  describes  how  the  researcher  interpreted  the  scaled  responses. 

Statement  25a  -  “  I  believe  outside  investment  by  a  third  party  is  proof  of  the 
commercial  potential  of  a  research  project”. 

1.  Statement  25a  Graphic  Representation. 

See  Figure  4.6  on  following  page. 

2.  Statement  25a  Narrative  Analysis. 

The  rationale  for  developing  this  statement  was  predicated  on  whether  or  not 
outside  investment  truly  provides  insight  into  a  firm’s  ability  to  market  its  potential 
products  thereby  enhancing  prospective  sales.  If  a  firm  agreed  with  the  statement,  then 
we  could  infer  that  the  firm  would  place  a  significant  value  in  marketing  their  products 
throughout  the  research  process  to  realize  a  faster  return  on  investment  for  any 
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forthcoming  developments.  If  a  firm  disagreed,  we  could  infer  that  they  did  not  place 
value  in  obtaining  outside  investment  funding. 


Figure  4.6  Frequency  of  Response  to  Statement  25a. 
(Source:  Designed  by  the  researcher) 


Mean  Score:  3.1351 

Standard  Deviation:  0.822 

Cl  Range:  2.9587-3.31 16 

The  pattern  of  the  oral  and  written  comments  tended  to  support  both  sides  of  the 
argument.  15  of  the  37  respondents  that  disagreed  with  the  statement  cited  a  belief  that 
outside  investment  in  a  SBIR  firm  during  Phase  I  and  II  constituted  a  “proposal  strategy” 
that  served  to  gain  preferential  treatment  via  inclusion  in  the  Fast  Track  program  during 
the  contract  evaluation  phase.  Key  words  such  as  “manipulation”  and  “temporary  loans" 
were  used  when  describing  how  competing  firms  could  make  proposals  look  more 
promising  on  paper.  One  firm  provided  a  detailed  example  where  people  could  make 
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clandestine  agreements  with  the  so-called  “investors”  on  a  business  loan  and  later  pay  the 
loan  back  to  the  “investor”  once  the  Phase  II  funding  is  approved. 

Four  firms  noted  that  audit  procedures  do  not  seem  foolproof.  In  this  situation, 
the  “outside  investment”  is  not  truly  an  investment,  it  is  simply  a  “temporary  loan”  to 
make  it  comply  with  Fast  Track  requirement,  but  in  reality  no  “real”  money  is  spent  on 
the  project. 

The  opposing  view  supported  by  10  of  the  37  firms  cited  the  commitment  of 
currency  as  the  only  method  to  effectively  determine  if  a  concept  was  worth  exploring. 
Although  none  of  the  firms  would  commit  to  an  extreme  view  of  “Strongly  Agree”  or 
Strongly  Disagree”,  their  approach  to  this  question  dictated  that  determining  a  consensus 
on  this  issue  would  be  difficult. 

Statement  25b  -  ‘T  would  change  a  superior  technical  research  approach  to  a 
slightly  lesser  technical  research  approach  if  I  could  gain  access  to  outside  investment” 

3.  Statement  25b  Graphic  Representation 

See  Figure  4.7  on  the  following  page. 

4..  Statement  25b  Narrative  Analysis 

The  rationale  for  developing  this  statement  was  to 'determine  if  outside  investment 
shaped  the  proposed  research  approach.  If  a  firm  agreed  with  die  statement,  then  we 
could  infer  that  they  placed  a  greater  value  on  potential  commercial  sales  opportunities  to 
non-DoD  customers  through  the  development  of  dual  use  technologies.  Disagreement 
would  suggest  that  firms  thought  their  optimal  proposed  solution  could  be  compromised 
by  outside  investor  influence. 
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Figure  4.7  Frequency  of  Response  to  Statement  25b. 

(Source:  Designed  by  the  researcher) 

Mean  Score:  2.7838 

Standard  Deviation:  1.0836 

Cl  Range:  2.5512-  3.0164 

The  variance  in  responses  for  this  technical  approach  question  suggests  that  each 
firm  decides  a  proposed  course  of  action  after  careful  examination  of  its  own  business 
considerations.  Once  internal  and  external  factors  are  evaluated,  companies  will  have  to 
provide  a  balanced  proposal  that  addresses  both  technical  merit  and  the  potential  for 
commercialization. 

The  key  point  for  program  executors  is  to  determine  how  best  to  structure 
solicitations  to  mitigate  the  influences  of  this  decision  making  process  while  completely 
fulfilling  agency  specific  research  requirements.  This  dilemma  represents  an  inherent 
conflict  within  the  program  because  awards  are  required  to  support  research  that  meets 
the  needs  of  the  awarding  agencies,  yet  products  developed  are  also  expected  to  be 
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successful  in  the  commercial  marketplace.  Eight  of  the  14  firms  that  responded  with 
“Strongly  Agree”  or  “Agree”  cited  this  dilemma  as  a  central  tenant  in  their  response. 
They  indicated  that  business  objectives  forced  them  to  utilize  this  approach  as  a  means  of 
maintaining  competitiveness  within  their  respective  areas  of  expertise. 

Seven  of  the  12  firms  surveyed  responded  that  they  disagreed  with  the  statement 
because  of  their  belief  that  outside  investors  would  influence  the  process  to  such  a  high 
degree  that  the  final  proposal  might  be  evaluated  as  more  than  “slightly  less”  than 
optimal. 

Statement  25c  -  “Raising  the  interest  of  outside  investors  for  SBIR  research 
topics  is  challenging.” 

5.  Statement  25c  Graphic  Representation. 


Strongly  Agree  Neither  Disagree  Strongly 
Agree  Agree  nor  Disagree 

Disagree 


Figure  4.8  Frequency  of  Response  to  Statement  25c. 
(Source:  Designed  by  the  researcher) 

Mean  Score:  1.7297 

Standard  Deviation:  .9324 
Cl  Range:  1.5296-1.9298 
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6.  Statement  25c  Narrative  Analysis. 

The  rational  for  developing  this  statement  was  that  if  respondents  agreed  with  this 
assertion,  then  an  argument  could  be  made  that  Army  SBIR  topics  were  considered 
abstract  and  provided  little  commercial  potential.  Low  levels  of  additional 
developmental  funding  and  sales  could  potentially  be  subsequent  outcomes  of  this 
reduced  commercial  potential. 

Based  on  the  low  mean  score  of  1.7297  and  the  written  and  oral  comments,  the 
researcher  surmises  that  there  is  a  strong  belief  within  the  SBIR  firms  that  SBIR  projects 
are  too  narrowly  constructed  to  appeal  to  outside  investors  seeking  to  exploit  commercial 
applications.  Although  the  program  is  satisfying  Army  research  objectives,  the  data 
indicate  that  the  SBIR  program  is  continuing  to  fall  short  of  commercial  market 
expectations. 

Eight  of  the  32  firms  that  responded  with  “Strongly  Agree”  or  “Agree”  noted  that 
only  a  very  small  commercial  market  existed  for  the  technologies  they  received  an  award 
to  study.  Four  firms  noted  persistent  problems  with  attracting  capital.  This  was 
attributed  to  low  commercialization  prospects  and  the  perception  of  low  profitability  with 
DoD  contracts.  Finally,  One  firm  noted  that  they  had  tried  to  meet  with  a  large  firm  in 
the  same  industry  and  could  not  because  of  fears  of  patent  issues. 

F.  SBIR  PROGRAM  MANAGER  INTERVIEW 

The  researcher  conducted  a  personal  interview  with  Dr.  Kenneth  Bannister,  Army 
SBIR  program  manager,  to  ascertain  how  key  policy  makers  viewed  program  execution. 
The  interviews  were  conducted  during  the  April  2001  Spring  SBIR  Conference  in  Crystal 
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City,  Virginia.  The  purpose  was  to  determine  what,  if  any,  steps  were  necessary  to 
optimize  the  SBIR  process. 

When  asked  about  sharing  project  results  across  the  DoD/Govemment  spectrum, 
Banister  commented  that  SBIR  results  were  published  in  booklets  and  brochures  that  are 
subsequently  distributed  throughout  the  business  community  via  conferences, 
organizational  websites,  and  the  Defense  Technical  Information  Center  (DTIC).  (Ref  20) 
He  believes  the  effectiveness  of  these  efforts  to  publicize  the  benefits  of  the  program  are 
commensurate  to  those  used  by  other  industries  to  promote  their  innovative  products. 

Bannister  (Ref  20)  acknowledged  that  reporting  quantifiable  successes  remains  a 
dilemma  for  there  is  no  formal  process  to  gain  empirical  evidence  to  support  the  degree 
of  successful  commercialization  within  the  program.  More  importantly,  he  noted  that 
such  a  requirement  does  not  exist,  either  in  statute  or  policy,  for  companies  to  provide 
such  data.  This  failure  to  gather  empirical  evidence  and  a  reliance  upon  anecdotal 
methods  such  as  media  stories  and  “word  of  mouth”  may  serve  to  hinder  the 
organization’s  ability  to  comply  with  the  annual  performance  reporting  requirements 
required  by  GPRA  provisions  and  the  2000  SBIR  program  reauthorization. 

G.  SBIR  FIRM  RECOMMENDATIONS  FOR  IMPROVEMENT 

The  researcher  found  that  this  study  provided  valuable  insight  into  the  factors  that 
affect  proposal  construction.  One  of  the  most  illuminating  sections  of  the  survey  was  the 
follow-up  dialog  between  the  researcher  and  the  surveyed  firms.  The  Army  is  fortunate 
to  have  such  a  motivated  base  of  research  firms  competing  to  provide  ground-breaking 
technologies  to  solve  emerging  requirements.  Some  specific  comments  from  firms 
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indicate  opportunities  for  improvement  within  the  program.  Firms  provided  the 
following  responses  when  asked  how  they  would  improve  the  program: 

“Reduce  the  focus  of  outside  investment.” 

“Focus  research  on  areas  that  lend  themselves  to  commercial  products.” 

“Increase  the  contract  award  amount  to  keep  pace  with  business  cost.” 

H.  SURVEY  DESIGN  FEEDBACK 

Subsequent  review  of  the  data  provided  evidence  that  ten  questions  on  the  original 
survey  instrument  used  in  this  study  to  be  of  little  use.  Overall,  the  firms  surveyed  were 
reluctant  to  reveal  personal  information  about  matters  such  as  the  number  of  founders, 
their  backgrounds,  and  when  the  firm  was  started.  Additionally,  questions  that  involved 
estimates  from  technical  personnel,  such  as  expected  sales,  returned  a  series  of  vague 
responses  that  did  not  add  value  to  the  project  nor  would  have  any  tangible  impact  on 
future  program  assessments 

I.  SUMMARY 

This  chapter  presented  the  survey  data  and  provided  an  analysis  of  their  respective 
impacts  on  the  program.  Summarized  results  of  key  questions  were  provided  along  with 
a  narrative  interpretation  of  what  the  results  mean  in  terms  of  sales,  marketing  strategy, 
customer  type,  and  types  of  products  produced. 

Based  on  the  survey  results  it  is  clear  that  the  program  is  successful  in  producing 
a  return  on  investment  that  is  substantially  higher  than  the  DoD  average;  that  the  DoD  is 
the  primary  beneficiary  of  the  Army  SBIR  research  efforts;  that  research  firms  would  not 
be  involved  in  Army  SBIR  efforts  if  the  program  were  to  cease  it  existence;  and  that 
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solicitation  evaluation  procedures  do  impact  each  research  firm’s  approach  to  proposal 
development. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION. 

“There  is  great  room  for  upward  change.”  (Ref  26) 


This  chapter  provides  the  conclusions  and  recommendations  for  the  data  collected 
from  SBIR  participants.  37  research  firms  responded  to  25  closed  end  questions.  The 
results  of  the  survey  are  used  to  answer  the  primary  and  secondary  research  questions 
outlined  in  Chapter  I. 

B.  CONCLUSIONS 

The  Army  is  successfully  achieving  its  SBIR  program  commercialization  goals. 
The  average  sales  per  award  clearly  indicate  that  the  Army  projects  surveyed  are 
outperforming  both  DoD  and  Fast  Track  award  averages. 

Firms  receiving  SBIR  funding  are  producing  useful  products  that  fulfill  DoD 
requirements.  The  DoD  consumption  of  85%  of  Army  SBIR  results  indicate  that  Army 
SBIR  solicitations  are  filling  vital  DoD  research  requirements  that  may  not  otherwise  be 
satisfied 

C.  RECOMMENDATIONS 

This  survey  provides  several  lessons  about  how  firms  view  the  SBIR  program  and 
how  those  views  impact  their  approach  to  each  solicitation.  The  following  are 
recommendations  to  assist  ARO-W  (A)  in  capitalizing  on  each  firms’  already  strong 
desire  to  participate  in  the  program.  They  also  provide  a  mechanism  for  ARO-W  (A)  to 
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collect  the  requisite  data  to  fulfill  reporting  requirements  mandated  by  the  Government 
Performance  and  Results  Act. 

1.  Minimize  the  Emphasis  on  Commercialization 

The  empirical  data  clearly  proves  that  DoD  is  the  main  consumer  of  Army  SBIR 
technological  developments.  Reducing  the  commercialization  evaluation  factor  weight 
will  encourage  firms  to  provide  the  best  approaches  to  Army  research  requirements. 

2.  Conduct  Regular  Program  Surveys 

The  data  to  gauge  program  effectiveness  is  available,  but  a  regular  mechanism  is 
needed  to  capture  it.  The  collection  of  this  data  will  be  helpful  in  preparing  annual 
performance  reports  for  GPRA  reporting  requirements.  An  electronic  version  of  the 
proposed  survey  instrument  in  Appendix  B  could  be  used  to  perform  this  task. 

Additionally,  it  is  recommended  that  contracting  officer  technical  representatives 
at  Army  Labs  and  RDECs  conduct  the  survey.  They  will  have  a  working  relationship 
with  the  performing  research  firms  that  can  enhance  the  quality  of  the  collected  data. 

3.  Use  a  Small  Survey  Population  to  Prepare  Program  Results 

Because  the  program  is  relatively  stable,  a  small  sample  size  will  be  sufficient  to 
achieve  a  level  of  certainty  required  to  present  a  suitable  projection  of  program  benefits. 
The  key  point  to  remember  is  that  this  program  accounts  for  only  2.5%  of  an  agency’s 
extramural  budget  in  excess  of  $100  million.  Since  we  do  not  provide  as  much  scrutiny 
to  large  businesses  with  respect  to  how  well  they  have  commercialized  their  Federally 
sponsored  research  funding,  the  level  of  confidence  needed  to  fulfill  the  GPRA  report 
requirement  should  weigh  the  estimated  costs  of  data  collection  against  the  projected 
benefits  derived  from  such  a  report. 
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4.  Modify  the  Survey  Instrument  to  Enhance  Responsiveness 

The  researcher  recommends  modifying  the  survey  instrument  to  reflect  questions 
of  greatest  impact  for  assessing  program  success.  Reducing  the  number  of  questions  and 
simplifying  the  survey  could  enhance  future  response  rates.  A  sample  survey  reflecting 
the  questions  of  greatest  impact  is  enclosed  in  Appendix  B. 

D.  ANSWERS  TO  RESEARCH  QUESTIONS 

1.  Primary  Research  Questions 

How  does  the  Army  define  and  measure  Small  Business  Innovation  Research 
(SBIR)  Program  success? 

The  Army  defines  program  success  as  complying  with  DoD’s  four  stated  goals; 
1)  stimulating  technological  innovation,  2)  using  small  businesses  to  meet  Federal 
research  and  development  needs,  3)  fostering  participation  of  minority  and  disadvantaged 
firms  in  technological  innovation,  and  4)  increasing  private-sector  commercialization  of 
innovations  derived  from  Federal  research.  It  further  defines  program  success  as  funding 
research  proposals  that  provide  the  best  technological  solutions  to  Army  research  needs. 

The  relative  measure  for  success  is  a  function  of  funding  all  high  quality 
proposals  until  program  funds  are  fully  expended. 

2.  Subsidiary  Research  Questions 

•  What  are  the  Small  Business  Innovation  Research  (SBIR)  and  Fast  Track 
Programs? 

The  SBIR  program  is  a  set-aside  effort  that  allows  small  businesses  to  participate 
in  Federal  research  initiatives.  The  Fast  Track  program  is  an  expedited  proposal  review 
process  that  provides  preferential  treatment  to  firms  that  secure  outside  investment  to 
augment  Government  provided  research  funding. 
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•  What  are  the  stated  goals  of  the  DoD  SBIR  Program? 

The  stated  goals  of  the  DoD  SBIR  program  are  to: 

a)  stimulate  technological  innovation; 

b)  use  small  business  to  meet  Federal  research  and  development 

needs; 

c)  foster  and  encourage  participation  by  minority  and  disadvantaged 
persons  in  technological  innovation;  and 

d)  increase  private-sector  commercialization  of  innovations  derived 
from  Federal  research  and  development. 

•  What  are  the  various  laws,  regulations,  and  policies  that  might  affect  the  SBIR 
and  Fast  Track  Programs? 

The  Small  Business  Innovation  Development  Act  in  1982  provided  the  foundation 
for  the  SBIR  program.  Successive  reauthorizations  via  the  Small  Business  Research  and 
Development  Enhancement  Act  of  1992  and  2000  have  added  the  commercialization 
tenet  that  currently  dominates  the  focus  of  the  program. 

•  What  major  products  are  created  through  the  Army  SBIR  program  and  how 
does  the  Department  of  Defense  integrate  those  products  into  its  inventory? 

The  Army  SBIR  program  requirements  cover  the  full  spectrum  of  research 
solutions.  Software  applications,  intermediate  and  final  hardware  components,  process 
technologies,  and  new  or  improved  service  capabilities  are  the  major  products  of  the 
SBIR  program. 

Products  are  integrated  into  the  inventory  through  acquisition  of  the  technology  or 
final  product.  The  Army  seeks  to  accomplish  this  task  by  linking  the  contractor  with  the 
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Army  Lab  or  RDEC  responsible  for  developing  the  requirement.  This  linkage  ensures 
that  future  related  research  efforts  will  have  some  intrinsic  knowledge  of  past  SBIR 
projects. 

•  How  does  the  Department  of  the  Army  track  the  commercialization  of  SBIR 
products  through  die  program’s  life  cycle? 

It  does  not  have  a  formal  mechanism  to  accomplish  this  task.  The  Army  SBIR 
effort  relies  on  “word  of  mouth”  success  stories  or  media  exposure  to  gauge  Phase  III 
success. 

•  Do  current  Department  of  the  Army  requirements  generation,  marketing, 
solicitation,  proposal  evaluation,  source  selection  and  surveillance  procedures 
lead  to  enhanced  program  success  and  greater  commercialization  of  products? 

The  answer  is  yes  and  no.  Congress  has  determined  that  the  program  is 
successful  in  accomplishing  its  program  goals,  but  there  are  no  recurrent  empirical 
measures  through  which  the  Army,  or  any  other  agency,  can  readily  use  to  substantiate 
that  statement. 

•  What  changes  are  needed  to  the  Department  of  the  Army’s  SBIR  Program  to 
enhance  the  program’s  performance  as  it  relates  to  stated  goals? 

A  review  of  the  evaluation  factors  to  support  superior  technical  approaches  and  to 
minimize  the  impact  of  outside  investment  is  recommended  to  enhance  program 
execution. 

E.  AREAS  OF  FURTHER  RESEARCH 

There  are  still  several  issues  that  can  be  addressed  in  further  detail.  While  the 
data  show  that  the  program  is  clearly  successful  from  a  sales  perspective,  the  ability  to 
quantitatively  measure  how  well  other  program  goals  are  fulfilled  is  a  problem.  The 
researcher  recommends  further  study  to  determine  how  program  participation  by 
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disadvantaged  and  minority  firms  could  be  increased.  Additionally,  further  study  is 
needed  to  determine  how  Army  solicitation  requirements  can  be  drafted  to  capitalize  on 
dual  use  applications  that  could  enhance  the  commercialization  potential  of  future  SBIR 
awards  and  products. 
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A  AM. 

Army  SBIR  Award  Survey  Questions 

Proposal  Title: 

Principle  Investigator: 

Phase  II  Contract  #: 

1.  Is  the  information  listed  above  correct? 

Yes.  Go  to  Question  2 

No.  Provide  correct  information  below  then  proceed  to  Question  2. 

Title: 

Principle  Investigator: 

Phase  II  Contract  Number: 

2.  What  is  the  current  status  of  this  SBIR  Project?  (Check  one) 

Project  has  not  yet  completed  Phase  II. 

Project  completed  Phase  II. 

Project  was  not  awarded  a  Phase  II. 

Don’t  Know 

3.  How  do  you  expect  to  commercialize  your  SBIR  award?  (Check  one) 

_ No  Commercial  Product,  Process,  or  Service  is  Planned 

_ Software 

_ Intermediate  Hardware  Product  or  Component 

_ Final  Hardware  Product 

_ Process  Technology 

_ New  or  Improved  Service  Capability 

_ Don’t  Know 


Question  4  concerns  whether  the  project  is  still  active  or  if  and  when  it  was  dropped. 


4.  When  did  your  company  drop  the  project?  (Check  one) 

Still  Active,  not  dropped. 
Don’t  Know 

More  than  one  year  after  completing  Phase  II. 


During  or  at  the  end  of  Phase  II 
Within  one  year  after  completing  Phase  II. 
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If  the  technology  developed  during  this  project  has  led  to  no  additional  developmental 
funding  or  sales  (and  neither  of  these  is  expected  to  occur),  skip  to  Question  16. 


Additional  Developmental  Funding  and  Sales 

PLEASE  READ  THE  FOLLOWING  DEFINITIONS 

Additional  Developmental  Funds:  Include  funds  from  federal  or  private  sector  sources,  or 
from  your  own  company,  used  for  further  development  of  the  technology  develop  during  the 
Phase  II  project. 

Sales:  All  sales  of  a  product,  process,  or  service,  to  Federal -nr  Private  Sector  Customers 
resulting  from  the  technology  developed  during  this  Phase  II  project. 

(Includes  the  sale  of  technology  or  rights,  etc.) 


5.  To  date,  what  has  been  the  total  additional  developmental  funding  for  the  technology 
developed  during  this  project? 

(Do  not  include  related  SBIR  funds  received from  DoD  or  other  federal  agencies.  Enter  dollars 
provided  by  each  of  the  following  sources.  If  none,  enter  zero.) 


Sources 

a.  Non- SBIR  Federal  Funds 

b.  Your  Own  Company 

c.  Other  Private  Company 

d.  U.S.  Venture  Capital  Institution 

e.  Foreign  Venture  Capital  Institution 

f.  Private  Investor 

g.  Personal  Funds 

h.  State  or  Local  Governments 

i.  College  or  Universities 

j.  Other  Sources  (Specify) 


Dollars 


Dt  n’t  Know 
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your  company’s  activities  with  other  companies  and  investors? 

(Check  all  that  apply) 


Ongoing  Finalized 

Negotiations  Agreements 

Activities  I - ■  ■ - ■ 

a.  Licensing  Agreement 

b.  Sale  of  Complete  Ownership  _  _ 

c.  Sale  of  Partial  Ownership  _  _ 

d.  Sale  of  Technology  Rights  _  _ 

e.  Joint  Venture  Agreement  _  _ 

f.  Marketing/Distribution  Agreement  _  _ 

g.  Manufacturing  Agreement  _  _ 

h.  Other  _  _ 

i.  Don’t  know  I _ I  I _ 


7.  Which  of  the  following,  if  any,  describes  the  type  and  status  of  marketing  activities  by  your 
company  and /  or  your  licensee  for  this  project? 

(Check  only  one  block  for  each  activity) 


Marketing  Activity  Planned 

a.  Preparation  of  Marketing 

b.  Hiring  of  Marketing  Staff 

c.  Publicity/Advertising 

d.  Test  Marketing 

e.  Other  (Specify): 


Not  Don’t 

Underway  Completed  Needed  Know 


8.  Has  your  company  and  /or  licensee  had  any  actual  sales  of  products,  processes,  services  or 
other  sales  from  the  technology  developed  during  this  project?  (Check  all  that  apply.) 


a.  Sales  of  Product(s) 

b.  Sales  of  Process(es) 

c.  Sales  of  Services(s) 

d.  Other  Sales 

(e.g.,  rights  to  technology,  etc) 

e.  No  Sales  to  Date 


►  Skip  to  Question  1 1 
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approximate  amount  of  total  sales  resulting  from  the  technology  developed  during  this  project? 
If  multiple  SBIR  projects  contributed  to  the  ultimate  product,  report  only  the  share  of  total  sales 
appropriate  to  this  SBIR  project. 

(Enter  dollars [$].  If  none,  enter  0 [zero].) 


Year  when  First  Sale  Occurred 


Total  Sales  Dollars  of  Product(s) 
Process(es)  or  Service(s)  to  Date 

Other  Total  Sales  Dollars 

(e.g..  Rights  to  technology,  etc.)  to  Date 


10.  To  date,  what  percent  of  total  sales  from  the  technology  developed  during  this  project  have 
gone  to  the  following  customers? 

(If  none,  enter  0 [zero].  Round  percentages.  Answers  should  add  to  100%) 


Private  Sector 

Department  of  Defense  (DoD) 
Prime  Contractors  for  DoD 
NASA 

Other  Federal  Agencies 
State  or  Local  Governments 
Export  Markets 
Other  (Specify) 

Don’t  Know 


1 1.  Expected  Sales.  For  your  company  and/or  your  licensee,  what  is  the  approximate  amount  of 
total  sales  expected  between  now  and  the  end  of 2002  resulting  from  technology  developed 
during  this  project?  (If  none,  enter  0  [zero]) 

If  no  sales  to  date,  what  year  do  your  expect  your  first  sale? 

IZZ1  2001  2002  I  1 2003  □  Later  than  2003 


Total  Sales  Dollars  of  Product  (s),  Process(es)  or  Services(s) 
expected  between  now  and  the  end  of 2002. 

Other  Total  Sales  Dollars  (e.g.,  rights  to  technology,  ect.) 
expected  between  now  and  the  end  of  2002. 

□  Don’t  Know 
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Program? 


Yes 

No  1  ►  Skip  to  Question  1 5 

Don’t  Know 


Questions  13  &  14  apply  to  Fast  Track  Participants 

13.  What  sources  provided  matching  or  co-investment  funding  that  enabled  your  company  to 
apply  for  Phase  II  under  Fast  Track? 

(Check  all  that  apply) 

a.  Venture  Capital  Firm  Provided  Funding. 

b.  Another  Company  Provided  Funding. 

c.  Another  Company  Provided  Facilities,  r_ 

Equipment  and/or  Other  in  Kind  Support.  I _ 

d.  Own  Company  Provided  Funding. 

e.  Don’t  Know 

14.  How  long  in  months  did  it  take  to  obtain  and  finalize  Agreements(s)  for  third  party 
funding/in  kind  support? 

Months 


15.  Prior  to  your  SBIR  award,  did  your  company  receive  funds  for  research  or  development  of 
the  technology  in  this  project  from  any  of  the  following  sources? 

(Check  all  that  apply) 

Prior  SBIR 

Prior  non-SBIR  Federal  R&D 
Venture  Capital 
Other  Private  Company 
Private  Investor 


Internal  Company  Investment 
State  or  Local  Government 
College  or  University 
Don’t  Know 
No 
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16.  In  your  opinion,  in  the  absence  of  this  SBIR  award,  would  your  company  have  undertaken 
this  project?  (Check  one) 

_ Definitely  Not 

_ Probably  Not 

_ Uncertain 

_ Probably  Yes 

_ Definitely  Yes 

17.  What  year  was  your  company  founded? 

18.  Information  on  company  founders. 

a.  Number  of  founders. 

b.  Number  of  founders  with  a  business  background. 

c.  Don’t  Know 


19.  Most  recent  employment  of  founders  prior  to  founding  this  company? 
(Check  all  that  apply) 


Other  Private  Company 
College  or  University 


Government 
Don’t  Know 


20.  Total  revenue  for  the  company  during  your  fiscal  year  (or  calendar  year)  2000. 
(Check  One) 

Less  than  $100,000 

_ $100,000  to  $499,999 

_ $500,000  to  $999,000 

_ $1,000,000  to  $4,999,999 

_ $5,000,000  to  $19,999,999 

_ More  than  $20,000,000 
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award? 


Prior  Phase  I  awards 
Prior  Phase  II  awards 
Don’t  Know 


22.  Please  give  the  number  of  patents,  copyrights,  and/or  scientific  publications  for  the 
technology  developed  during  this  project. 

(Enter  number.  If  none,  enter  0  [zero]) 

Number  Applied  for/Submitted  Number  Received/Published 

Patents(s)  L___J  L____ 

Scientific  Publications  | _ |  _ 

Don’t  Know 


23.  Which,  if  any,  of  the  following  has  your  company  experienced  as  a  result  of  the  technology 
developed  during  this  project? 

(Check  all  that  apply) 


-  Made  an  initial  public  stock  offering  in 
Planned  an  initial  public  stock  offering. 
Established  one  or  more  spin-off  companies. 
None  of  the  above. 

Don’t  Know 


(enter  year) 


24.  Employee  Information. 

(Enter  the  number  of  employees) 

Number  of  employees  when  Phase  II  began 

Current  number  of  employees 

_ Number  of  employees  hired  as  a  direct  result  of  the  technology  developed 

during  this  Phase  II  project. 


S  "  ”  '  -O  ■T  J  x  \ - - y - - -  "  ~  j  ~  • - 

a.  I  believe  outside  investment  by  a  third  party  is  proof  of  the  commercial  potential  of  a 
research  project. 

Strongly  Agree 
Agree 

Neither  agree  nor  disagree 
Disagree 

Strongly  Disagree 

b.  I  would  change  a  superior  technical  research  approach  to  a  slightly  lesser  research 
approach  if  I  could  gain  access  to  outside  investment. 

_ Strongly  Agree 

_ Agree 

_ Neither  agree  nor  disagree 

_ Disagree 

_ J  Strongly  Disagree 

c.  Raising  the  interest  of  outside  investors  for  SBIR  research  topics  is  challenging. 

Strongly  Agree 
_ Agree 

_ Neither  agree  nor  disagree 

_ Disagree 

_ I  Strongly  Disagree 

26.  Please  provide  the  following  information  about  the  person  who  completed  this 
questionnaire. 

Name: 

Title: 

Telephone: 

E-Mail: 

Company  URL: 


27.  Would  you  like  an  e-mail  version  of  the  final  report? 


Yes 


No 
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Army  SBIR  Award  Survey 

Title: 

Principle  Investigator: 

Phase  II  Contract  #: 

1 .  Is  the  information  listed  above  correct? 

Yes.  Go  to  Question  2 

No.  Provide  correct  information  below  then  proceed  to  Question  2. 
Title: 

Principle  Investigator: 

Phase  II  Contract  Number: 

2.  How  do  you  expect  to  commercialize  your  SBIR  award?  (Check  one) 

_ No  Commercial  Product,  Process,  or  Service  is  Planned 

_ Software 

_ Intermediate  Hardware  Product  or  Component 

_ Final  Hardware  Product 

_ Process  Technology 

_ New  or  Improved  Service  Capability 

_ Don’t  Know 


Question  3  concerns  whether  the  project  is  still  active  or  if  and  when  it  was  dropped. 

3.  When  did  your  company  drop  the  project?  (Check  one) 

During  or  at  the  end  of  Phase  II 
Within  one  year  after  completing  Phase  II. 

More  than  one  year  after  completing  Phase  II. 


Still  Active,  not  dropped. 
Don’t  Know 
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If  the  technology  developed  during  this  project  has  led  to  no  additional  developmental 
funding  or  sales  (and  neither  of  these  is  expected  to  occur),  skip  to  Question  16. 


Additional  Developmental  Funding  and  Sales 

PLEASE  READ  THE  FOLLOWING  DEFINITIONS 

Additional  Developmental  Funds:  Include  funds  from  federal  or  private  sector  sources,  or 
from  your  own  company,  used  for  further  development  of  the  technology  develop  during  the 
Phase  II  project 

Sales:  All  sales  of  a  product,  process,  or  service,  to  Federal  or  Private  Sector  Customers 
resulting  from  the  technology  developed  during  this  Phase  II  project. 

(Includes  the  sale  of  technology  or  rights,  etc.) 


4.  To  date,  what  has  been  the  total  additional  developmental  funding  for  the  technology 
developed  during  this  project? 

(Do  not  include  related  SBIR  funds  received  from  DoD  or  other  federal  agencies.  Enter  dollars 
provided  by  each  of  the  following  sources.  If  none,  enter  zero.) 


Sources 

a.  Non-SB IR  Federal  Funds 

b.  Your  Own  Company 

c.  Other  Private  Company 

d.  U.S.  Venture  Capital  Institution 

e.  Foreign  Venture  Capital  Institution 

f.  Private  Investor 

g.  Personal  Funds 

h.  State  or  Local  Governments 

i.  College  or  Universities 

j.  Other  Sources  (Specify) 


Dollars 


Don’t  Know 
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your  company’s  activities  with  other  companies  and  investors? 

(Check  all  that  apply) 


Ongoing  Finalized 

Negotiations  Agreements 

Activities  I - ,  I - , 

a.  Licensing  Agreement 

b.  Sale  of  Complete  Ownership  _ 

c.  Sale  of  Partial  Ownership  _  _ 

d.  Sale  of  Technology  Rights  _  _ 

e.  Joint  Venture  Agreement  _  _ 

f.  Marketing/Distribution  Agreement  _  _ 

g.  Manufacturing  Agreement  _  _ 

h.  Other  _  _ 

i.  Don’t  know  I _ I  I _ 


6.  Which  of  the  following,  if  any,  describes  the  type  and  status  of  marketing  activities  by  your 
company  and 1  or  your  licensee  for  this  project? 

( Check  only  one  block  for  each  activity) 

Not  Don’t 

Marketing  Activity  Planned  Underway  Completed  Needed  Know 


a.  Preparation  of  Marketing 

b.  Hiring  of  Marketing  Staff 

c.  Publicity/Advertising 

d.  Test  Marketing 

e.  Other  (Specify): 


7.  Has  your  company  and  /or  licensee  had  any  actual  sales  of  products,  processes,  services  or 
other  sales  from  die  technology  developed  during  this  project?  (Check  all  that  apply.) 


a.  Sales  of  Product(s) 

b.  Sales  of  Process(es) 

c.  Sales  of  Services(s) 

d.  Other  Sales 

(e.g.,  rights  to  technology,  etc) 

e.  No  Sales  to  Date 


►  Skip  to  Question  10 
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approximate  amount  of  total  sales  resulting  from  the  technology  developed  during  this  project? 
If  multiple  SBIR  projects  contributed  to  the  ultimate  product,  report  only  the  share  of  total  sales 
appropriate  to  this  SBIR  project. 

(Enter  dollars [$].  If  none,  enter  0 [zero].) 


Year  when  First  Sale  Occurred 


Total  Sales  Dollars  of  Product(s) 
Processes)  or  Service(s)  to  Date 

Other  Total  Sales  Dollars 

(e.g.,  Rights  to  technology,  etc.)  to  Date 


9.  To  date,  what  percent  of  total  sales  from  the  technology  developed  during  this  project  have 
gone  to  the  following  customers? 

(If  none,  enter  0 [zero].  Round  percentages.  Answers  should  add  to  100%) 

Private  Sector 

Department  of  Defense  (DoD) 

Prime  Contractors  for  DoD 

_  NASA 

Other  Federal  Agencies 
State  or  Local  Governments 

_  Export  Markets 

_  Other  (Specify) 

Don’t  Know 


10.  Prior  to  your  SBIR  award,  did  your  company  receive  funds  for  research  or  development  of 
the  technology  in  this  project  from  any  of  the  following  sources? 

(Check  all  that  apply) 


Prior  SBIR 

Prior  non-SBIR  Federal  R&D 
Venture  Capital 
Other  Private  Company 
Private  Investor 


Internal  Company  Investment 
State  or  Local  Government 
College  or  University 
Don’t  Know 
No 
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1 1 .  In  your  opinion,  in  the  absence  of  this  SBIR  award,  would  your  company  have  undertaken 
this  proj  ect?  (Check  one) 

Definitely  Not 
Probably  Not 
Uncertain 

_ Probably  Yes 

_ Definitely  Yes 

12.  Total  revenue  for  the  company  during  your  fiscal  year  (or  calendar  year)  2000. 

(Check  One) 

Less  than  $100,000 

_ $100,000  to  $499,999 

_ $500,000  to  $999,000 

_ $1,000,000  to  $4,999,999 

_ $5,000,000  to  $19,999,999 

More  than  $20,000,000 

13.  How  many  federal  agency  SBIR  awards  has  your  company  received  prior  to  this  Phase  II 
award? 

_ Prior  Phase  I  awards 

_ Prior  Phase  II  awards 

_ |  Don’t  Know 

14.  Please  give  the  number  of  patents,  copyrights,  and/or  scientific  publications  for  the 
technology  developed  during  this  project. 

(Enter  number.  If  none,  enter  0  [zero]) 

Number  Applied  for/Submitted  Number  Received/Published 

Patents(s)  L__J  L___ 

Scientific  Publications  | _ |  _ 

Don’t  Know 
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(Enter  the  number  of  employees) 

_ Number  of  employees  when  Phase  II  began 

_ I  Current  number  of  employees 

_ I  Number  of  employees  hired  as  a  direct  result  of  the  technology  developed 

during  this  Phase  II  project. 


16.  Please  provide  the  following  information  about  the  person  who  completed  this 
questionnaire. 

Name: 

Title: 

Telephone: 

E-Mail: 

Company  URL: 
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Current  SBIR  How  d°  «*** <°  When  did  you 

Project  Status  commercialize  your  drop  (he  project?  Non-SBIR  Additional  Developmental  Funding 
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Non-SBIR  Additional  Developmental 

Funding  Ongoing  Negotiations  Finalized  Agreements 


Marketing  Activity  Marketing  Activity  Marketing  Activity  Marketing  Activity  Marketing  Activities 
Planned  Underway  Completed  Not  Needed  Unknown 


Firm  26 
Firm  27 
Firm  28 


Marketing  Activity  Marketing  Activity  Marketing  Activity  Marketing  Activity  Marketing  Activities 
Planned  Underway  Completed  Not  Needed  Unknown 


Actual  Sales  Data  SBIR  Sales  Data  Percentage  of  Sales  of  SBIR 

Products 
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Percentage  of  Sales  of  SBIR  Products 


Fast  Track  Source  of  Fast  Track  Months  before  finalizing 

Application  Capital  agreements  Funding  Source  Prior  to  SBIR  Award 
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firm  have  undertaken  Fmjndpri  Founders  Most  Recent  Founder 

'A  w/o  SBIR  Funding  Information  Employment  Situation 
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Company  Experience  as  a  Result  of  Employee 

the  SBIR  Program  Information 
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